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Background/context: 

Research has shown that delays in language development often predict literacy problems for 
children in later grades. Low performance in early vocabulary acquisition has a detrimental 
impact on language and literacy competencies in later grades (Bracken, 2005; 

Cunningham, 1997; Dickinson & Tabors, 2001; NICHD Early Child Care Research Network, 
2005; Spira, Bracken, & Fischel, 2005), limited vocabulary skills plague poor children 
(Dickinson & Tabors, 2002), and the gap between poor and middle class children’s vocabularies 
increases over time (Harris & Herrington, 2006; Lee & Burkam, 2002). Several initiatives have 
formed with the goal of combating these problems and reversing this downward trend. 
Consequently, the United States has seen a surge in interventions targeting language and literacy 
development for preschool children over the past few decades. These interventions, as 
exemplified by the Preschool Curriculum Evaluation Research Initiative (PCER) funded by the 
Institute of Education Sciences (IES) in 2002 and the NCER report (Preschool Curriculum 
Evaluation Research Consortium, 2008) have had mixed results from showing substantial 
positive effects to having limited and at times negative influence on child outcomes. 

This paper presents the results of a meta-analysis whose aim was to broaden our understanding 
of various interventions focused on enhancing literacy development. Results will show the 
effectiveness of preschool curriculum targeting children from economically disadvantaged 
families. This synthesis also examines the ways in which the ethnic composition of preschool 
programs moderates the effect of the intervention. Additionally, it examines the moderating 
effects of preschool type (e.g. Head Start or other), parental involvement in literacy activities, 
and implementation of a mentoring program for teachers adopting the new curricula. 



Purpose/objective/research question/focus of study: 

The objective of this report is to review studies that report language and literacy outcomes 
associated with preschool curriculum-based interventions. Results from studies reporting on 
interventions targeting preschool children from low-income families were included regardless of 
the specific type of program. Although the majority of preschool interventions target classrooms 
affiliated with Head Start, studies with alternative programs were also considered. Results will 
help inform administrators and policy makers of the effectiveness of curriculum-based preschool 
interventions. The following research questions are highlighted: 

1. Do curriculum interventions have significant effects on preschool children’s vocabulary 
development? 

2. Are the effects of curriculum interventions on vocabulary affected by program 
characteristics? 
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Inclusion Criteria - Setting/Types of Studies: 

Studies were included in this synthesis if they met particular criteria. Intervention studies that 
employed random assignment with at least one treatment and one control group, and quality 
quasi-experimental studies were considered for inclusion. Control groups could take various 
forms and may include pre-existing curriculum or the same baseline curriculum as the treatment 
group without the additive/supplementary elements. Due to the nature of preschool programs 
and how children are assigned to teachers and classrooms, it was acceptable for researchers to 
assign at the classroom level. Studies had to have pre-test measures on vocabulary for all groups 
so that adjusted post-tests scores for treatment and control groups could be calculated. 
Additionally, only studies completed after 1990 were included, yet inclusion was not dependent 
on publication. Acceptable types of publications included journal articles, reports, conference 
presentations, and unpublished dissertations. A variety of publication types were represented in 
this analysis (see Table 1, Appendix B for details). 



Population/Participants/Subj ects : 

All studies that met the review criteria included samples with specific characteristics. All sample 
participants included children between the ages of 3 and 5 attending a preschool program. Only 
preschools that targeted children from low-income families were included; consequently, all 
preschool programs served children from high-poverty communities with high risk of academic 
failure. Head Start classrooms and programs receiving Title I funding automatically met this 
criterion as did other programs where the majority of the enrolled children were from low- 
income backgrounds. Cultural, linguistic, and racial backgrounds of the children in these studies 
differed. Studies that include second language learners in their sample were included. 



Intervention/Program/Practice : 

This meta-analysis focuses on the implementation of both comprehensive curricula and 
supplementary curricula with enhanced materials and/or instructional methods used in 
conjunction with pre-existing programs of instruction. Additionally, studies focused on language 
and literacy development with measurements targeting the effects of the curriculum on 
vocabulary acquisition. Furthermore, providers of the curriculum could vary between classroom 
teachers or others like support staff, specialists, researchers, and parents. It was necessary for the 
delivery of this intervention, however, to occur in the school setting. Nevertheless, classroom 
teachers delivered the majority of interventions. 



Research Design: 

A random effects model was used for this synthesis in order to best characterize variance 
between studies (Lipsey & Wilson, 2001). Effect sizes were calculated by finding the 
standardized mean difference while employing the Hedges’ g correction for sample size bias and 
inverse variance weights (Hedges & Olkin, 1985). Because random assignment was made at the 
classroom level and this synthesis reports effect sizes at the child-level, a standard error 
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adjustment was made to account for the influence of sample clustering (Hedges, 2007). Without 
this adjustment, standard errors would likely be inflated and the likelihood of statistical 
significance would be disproportionately greater. 



Data Collection and Analysis: 

The included studies reported on several dependent variables. This synthesis highlighted one 
major category of language and literacy competence: vocabulary. Studies eligible for inclusion 
reported vocabulary outcomes using either a receptive or expressive standardized, nationally- 
normed measure. If a study reported both receptive and expressive vocabulary outcomes, scores 
were taken from the receptive measurements (e.g. PPVT). It was not necessary for studies to 
directly report effect sizes for these outcomes; however, it was mandatory that effect sizes on 
these measures could be accurately calculated from the data reported. 



Findings/Results: 

To date, 17 studies originating from eight separate reports have been included in a synthesis; 
more studies will be added to the synthesis prior to the conference meeting. In addition, print 
awareness will be included as a literacy outcome in the final analysis. All studies included 
vocabulary outcomes; the majority of which reported pre and post scores of receptive vocabulary 
by administering the PPVT. In the cases when PPVT scores were not reported, data from 
expressive vocabulary assessments were used (e.g. EVT or EOWPVT). (See Table 1, Appendix 
B for details). 

A test of homogeneity was performed. Main effects analysis produced a significant Q-statistic 
(Q=64.23, DF=16, pc.Ol), allowing us to reject the null hypothesis that assumes homogeneity. 
Since the differences between studies appear to be caused by something other than sampling 
error, random effects analysis as well as moderator analysis was justified. 

Mean Effect Across Studies for Vocabulary Acquisition 

The overall effect size of preschool curriculum interventions on vocabulary was 0.066 (p>.05). 
The 95% confidence interval also indicates that this is not statistically significant [0.147,0.278]. 
These calculations signify that, overall, there are no significant effects of these interventions on 
the vocabulary outcomes of preschool children. The forest plot below (see Figure 1 in Appendix 
B) shows that the majority of interventions had little to no effect on vocabulary. This forest plot 
indicates that 6 of the 17 included studies have statistically significant effect sizes. Of those six, 
two have a negative impact, while the other four show positive effects. 

Because the Q-value (Q=64.231, DF=16, pc.Ol) was significant and allowed us to reject the null 
hypothesis that there is homogeneity among these studies, the mean effect size calculated above 
(Mean ES = .066, p>.05) is not representative of the overall effect and variability among 
preschool curricula. Likewise, the I-squared value (75.10) represents the portion of the total 
variance that is the between-study variance, which is not likely caused by chance. In this case 
roughly 75% of the total variance is represented by the between study variation; this is a 
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relatively large amount. These findings and the evidence that there exists a sizeable amount of 
variation between the 17 studies included in this synthesis give justification to explore moderator 
effects in effort to explain that variation. The ensuing moderator analysis helped to clarify which 
studies produced larger effects and which elements of those studies had the largest impact. 

Analysis of Impacts for subgroups of Studies on Vocabulary Outcomes 

In order to determine if sample and programmatic characteristics are systematically related to the 
effect sizes, an ANOVA within a random effects model was run on each moderator using a 
macro in SPSS (Wilson, 2005). This also signified whether the difference between categories 
within a group varied significantly. 

An ANOVA was also used to test whether the type of publication reporting the study moderated 
the effect of the curriculum intervention. No influence based on publication type was present (Q- 
between=.33, DF=1, p>.05). Therefore, it appears that publication type does not influence effect 
sizes, nor is there evidence of any publication bias. 

Moderator analysis did reveal some interesting significant results. The type of preschool 
program impacted the effect of the intervention. The Q-between value representing two types of 
programs (PkTyp), Head Start compared to programs not affiliated with Head Start, equaled 0.95 
(DF=1, pc. 10). (See Table 2, Appendix B for details). Interventions in Head Start programs had 
positive effects on children’s vocabulary development. The mean effects sizes between programs 
with a predominantly (over 50%) white enrollment and those with enrollments that were 
predominantly African American, Hispanic, or Asian/Pacific Islander (PredEth) were 
significantly different (Q-between=9.76, DF=1, pc.01). The mean effect size for programs with 
a non- White racial majority was significantly higher (.33) than that of programs with a White 
majority (-.53). 



Conclusions: 

One of the goals of this meta-analysis was to indicate the extent to which preschool curricular 
interventions increase vocabulary outcomes for children. Results of this synthesis showed that 
overall such interventions, some of which specifically target language and literacy development, 
have no significant effects on children’s vocabulary. The resulting effect size (ES=0.07) is quite 
small and non-significant. In practical terms, this effect size equates to a 1.05-point increase in 
the PPVT. Analysis of the FACES data (Zill, N., Resnick, G., & Kim, K., 2001) provided 
evidence that attending Head Start for a year raised children’s PPVT scores 4.3 points, on 
average. This increase equates to an effect size of 0.29. The curriculum interventions, in this 
synthesis, averaged about a 25% gain above and beyond that. One can conclude that attendance 
in a preschool program like that of Head Start has a larger positive effect on vocabulary 
acquisition than does the particular curriculum implemented in such a program. 

Moderator analysis, however, identified two primary differences between preschool programs. 
The type of preschool program impacted the effect of the intervention as results show Head Start 
programs had more positive effects on children’s vocabulary development. The influence of a 
preschool program’s ethnic and racial composition on the effectiveness of these interventions 
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was revealed. Results from an analysis of variance (ANOVA) indicated that interventions in 
programs predominantly serving children of color had positive effects on children’s vocabulary 
development. 

As is the case with all research, it is essential to use statistical analyses like the ones used in this 
synthesis to inform practice and policy. Certainly, the findings of this report are not enough to 
justify widespread policy regarding preschool interventions. However, evidence that curriculum- 
based interventions affect vocabulary gains differently with regard to the type of preschool 
program and the racial and ethnic background of the enrolled children suggests that a closer 
examination is warranted. Moreover, analysis of the effect of these interventions on additional 
outcomes (e.g. print awareness) is necessary, as several curricula included in this synthesis 
directly emphasize other elements of preschool children’s literacy and language development in 
addition to vocabulary acquisition. 
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Appendix B. Tables and Figures 

Table 1: Sample characteristics of 17 included studies 

Sample Characteristic N Sample Characteristic N 



Publication Type 




Total N (range) 




Journal Article 


11 


clOO 


6 


Report 


3 


100-199 


6 


Unpublished 


2 


200+ 


5 


Dissertation 


1 






Program Type 




Curriculum Type 




HS 


8 


Language & Literacy 


15 


T1 


2 


Math 


1 


UPK 


2 


Multiple 


1 


Public 


4 






Private 


1 






Measure of Vocabulary 




Program Predominant Ethnicity 




PPVT 


9 


White 


4 


EVT 


6 


African American 


2 


EOWPVT 


2 


Hispanic 


4 






Asian/Pacific Islander 


2 






No majority 


5 



Table 2: Relation of Moderators to Effect Size 



Moderator 


Group 


MeanES 


Q-between 


p-value 


Publication Type 


Journal Article (0) 


.115 


.33 


.57 




Other (1) 


-.011 






Type of Preschool* 


Head Start (0) 


.179 


2.97 


.08 




Other (1) 


-.001 






Pred Ethnicity*** 


White (0) 


-.525 


10.01 


.002 




Other (1) 


.329 






Parent Involvement 


No (0) 


.077 


.05 


.82 




Yes (1) 


.017 






Mentoring 


No (0) 


.009 


.23 


.63 




Yes (1) 


.113 







* pc. 10, ** pc. 05, *** pc.01 
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Figure 1. Random Main Effects Output 



Meta Analysis 



Study ID 



Statistics for each study 





Point 


Standard 




Lower 


Upper 








esti mate 


error 


Variance 


li mit 


limit 


2- Value 


p- Value 


1.000 


0.522 


0.212 


0.045 


0.106 


0.938 


2.461 


0.014 


4.000 


0.523 


0.260 


0.068 


0.013 


1.033 


2.009 


0.045 


8.000 


-0.218 


0.219 


0.048 


-0.647 


0.211 


-0.997 


0.319 


14.000 


0.110 


0.167 


0.028 


-0.217 


0.437 


0.659 


0.510 


15.000 


-0.232 


0.171 


0.029 


-0.566 


0.102 


-1.360 


0.174 


16.000 


0.119 


0.322 


0.104 


-0.513 


0.751 


0.369 


0.712 


17.000 


0.053 


0.218 


0.048 


-0.375 


0.481 


0.243 


0.808 


18.000 


-0.315 


0.169 


0.029 


-0.646 


0.016 


-1.866 


0.062 


24.000 


0.312 


0.169 


0.028 


-0.018 


0.642 


1.851 


0.064 


25.000 


-0.043 


0.212 


0.046 


-0.469 


0.373 


-0.203 


0.839 


26.000 


-0.041 


0.168 


0.028 


-0.371 


0.289 


-0.244 


0.807 


27.000 


0.060 


0.227 


0.052 


-0.385 


0.505 


0.264 


0.792 


28.000 


0.086 


0.329 


0.108 


-0.559 


0.731 


0.261 


0.794 


29.000 


0.998 


0.234 


0.055 


0.539 


1.467 


4.263 


0.000 


30.000 


0.762 


0.232 


0.054 


0.307 


1.217 


3.285 


0.001 


31.000 


•1.188 


0.382 


0.146 


-1.937 


-0.439 


•3.109 


0.002 


32.000 


-1.188 


0.436 


0.190 


-2.042 


-0.334 


-2.726 


0.006 




0.066 


0.109 


0.012 


-0.147 


0.278 


0.606 


0 544 



Point esti mate and 95% Cl 




Favours A Favours B 



Meta Analysis 
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